Text mining method and apparatus for extracting features of documents

ABSTRACT

Concerning feature extraction of documents in text mining, a method and an apparatus for extracting features having the same nature as those by LSA are provided that require smaller memory space and simpler program and apparatus than the apparatus for executing LSA. Features of each document are extracted by feature extracting acts on the basis of a term-document matrix updated by term-document updating acts and of a basis vector, spanning a space of effective features, calculated by basis vector calculations. Execution of respective acts is repeated until a predetermined requirement given by a user is satisfied.

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to Japanese Patent ApplicationNo. 2000-197421 filed on Jun. 29, 2000, the contents of which isincorporated herein by specific reference.

BACKGROUND OF THE INVENTION

[0002] 1. Field of the Invention

[0003] The present invention relates to a text mining method andapparatus for extracting features of documents. In particular, theinvention relates to a text mining method and apparatus for extractingfeatures of documents, wherein features are extracted such that allmutually associated documents and terms are placed near each other inthe feature space. Applications of the invention include document and/orweb retrieval, associated term retrieval, document classification.

[0004] 2. Description of the Related Art

[0005] In text mining as a technology for squeezing desired knowledge orinformation by making analysis of text data, effective featureextraction of documents is an important task for efficiently performingdocument and/or web retrieval, associated term retrieval, documentclassification and so on. As a typical document feature extractingmethod, a vector-space model as set out on page 313 of “Automatic TextProcessing” (Addison-Wesley, 1989) is frequently used.

[0006] In the vector-space model, when terms selected as indices in thedocuments, namely index terms representing the contents of thedocuments, are t in number, a vector V_(i) is used respectively tocorrespond to an index term T_(i) to define a t-dimensional vectorspace. All vectors forming thus the defined vector space can beexpressed as a linear combination of t in number of the vectorscorresponding to t in number of the index terms. In this vector space, adocument D_(r) is expressed as follows: $\begin{matrix}{D_{r} = {\sum\limits_{i = 1}^{i}{x_{ir}V_{i}}}} & (1)\end{matrix}$

[0007] In the foregoing expression (1), x_(ir) active on V_(i) is thecontribution of the index term T_(i) to the document D_(r) andrepresents a feature of the document. The feature is an amountrepresenting the term frequency of the index term in the document. Avector [x_(r1), x_(r2), . . . x_(rt)]′ of t×1 (t rows and one column)becomes a feature vector of the document D_(r). As the simplest case,when the index term T_(i) appears in the document D_(r), x_(ir) is setto 1. When the index term T_(i) does not appear in the document D_(r),x_(ir) is set to 0. In a more complicated case, as set forth in theforegoing publication on page 279 to 280, two quantities are used. Thesetwo quantities are a term frequency tf_(ri) of the index term T_(i) inthe document D_(r) and a document frequency df_(i) of documentscontaining the index term T_(i) in all documents registered in thedocument database.

[0008] For the group of documents consisting of d in number ofdocuments, a t×d term-document matrix X can be defined as follows:

X=[x₁ , x ₂ , . . . , x _(d)]

[0009] Here, a t-dimensional vector x_(j)=[x_(j1), x_(j2), . . . ,x_(jt)]′ expresses the feature vector of the document D_(j), and ′(dash)represents matrix inversion.

[0010]FIG. 1 is an illustration showing one example of documents,translated from Japanese sentences, registered in a document database,where “ronin” is a romanized word meaning students who, having failed aschool entrance-exam of a particular academic year, are preparing forone next year. FIG. 2 is an illustration showing one example of aterm-document matrix taking the Kanji (Chinese) characters appearing onthe documents shown in FIG. 1 as index terms. Kanji terms are underlinedin FIG. 1. In FIG. 2, among a character string “let me know about”appearing in all of the documents 1 to 3, the Kanji term “know” ischecked off from the index terms. FIG. 3 is an illustration showing oneexample of an actual input question, translated from Japanese, from auser, where Kanji terms are underlined. If the index terms of FIG. 2 areused to express the question, the question can be expressed with theterm-document matrix shown in FIG. 4.

[0011] In general, when the vector-space model is used, similarity sim(D_(r), D_(s)) of two documents D_(r) and D_(s) can be expressed asfollows: $\begin{matrix}{{{sim}\left( {D_{r},D_{s}} \right)} = \frac{\sum\limits_{i = 1}^{t}{x_{ir}x_{is}}}{\sqrt{\sum\limits_{i = 1}^{t}{x_{ir}^{2}{\sum\limits_{i = 1}^{t}x_{is}^{2}}}}}} & (2)\end{matrix}$

[0012] When the similarity of the question and each document of FIG. 1is judged on the basis of the meaning of the question of FIG. 3, thequestion of FIG. 3 is the most similar to the document 3 of FIG. 1.However, using the feature vectors as shown in FIGS. 2 and 4, thesimilarity of each document of FIG. 1 and the question of FIG. 3 isrespectively sim(document 1, question)=0.5477, sim(document 2,question)=0.5477, sim(document 3, question)=0.5477. In short, all havethe same similarity.

[0013] As a solution for such a problem, a method called Latent SemanticAnalysis (LSA) was proposed in “Journal of the American Society forInformation Science” 1990, Vol. 41, No. 6, pp. 391 to 407. This methodextracts latent meaning of the documents on the basis of co-occurrencesof the terms and is significantly outstanding in terms of retrievingefficiency. Here, “co-occurrences of terms” represents a situation wherethe terms appear simultaneously in the same documents/statements.

[0014] The LSA extracts a latent semantic structure of the documents byperforming singular value decomposition (SVD) for the term-documentmatrix. In the obtained feature space, mutually associated documents andterms are located near each other. In a report placed in “BehaviorResearch Methods Instruments & Computers” (1991), Vol. 23, No. 2, pp.229 to 236, retrieval using the LSA indicates a result of 30% higherefficiency in comparison with the vector-space model. LSA will beexplained hereinafter in more detail.

[0015] In LSA, at first, singular value decomposition is performed forthe t×d term-document matrix X as set out below.

X=T₀ S ₀ D ₀′  (3)

[0016] Here, T₀ represents an orthogonal matrix of t×m, S₀ represents asquare diagonal matrix of m×m with taking m in number of the singularvalues as the diagonal elements and setting 0 to the other elements. D₀′represents an orthogonal matrix of m×d. In addition, let us assume that0≧d≧t, and arrange the orthogonal elements of S₀ in descending order.

[0017] Furthermore, in LSA, with respect to the feature vector x_(q) oft×1 of a document D_(q), the following conversion is performed to derivea LSA feature vector y_(q) of n×1;

y_(q)=S⁻¹T′x_(q)  (4)

[0018] Here, S is a square diagonal matrix of n×n taking thefirst to(n)th of the diagonal elements of S₀, and T is a matrix of t×n drawingthe first to (n)th columns of T₀.

[0019] As an example, results of singular value decomposition of theterm-document matrix shown in FIG. 2 are given below. The matrices T₀,S₀ and D₀ are expressed as follows: $T_{0} = \begin{bmatrix}0.1787 & {- 0.3162} & 0.3393 \\0.1787 & {- 0.3162} & 0.3393 \\0.1787 & {- 0.3162} & 0.3393 \\0.4314 & {- 0.3162} & {- 0.1405} \\0.4314 & {- 0.3162} & {- 0.1405} \\0.1787 & 0.3162 & 0.3393 \\0.1787 & 0.3162 & 0.3393 \\0.4314 & 0.3162 & {- 0.1405} \\0.4314 & 0.3162 & {- 0.1405} \\0.1787 & 0.3162 & 0.3393 \\0.2527 & 0.0000 & 0.4798\end{bmatrix}$ $S_{0} = \begin{bmatrix}2.7979 & 0 & 0 \\0 & 2.2361 & 0 \\0 & 0 & 1.4736\end{bmatrix}$ $D_{0} = \begin{bmatrix}0.5000 & {- 0.7071} & 0.5000 \\0.5000 & 0.7071 & 0.5000 \\0.7071 & 0.0000 & {- 0.7071}\end{bmatrix}$

[0020] Let us assume that the dimension t of the LSA feature vectors is2 and applying the foregoing expression (4) to each feature vector ofthe term-document matrix in FIG. 2. Then, the LSA feature vectors of thedocuments 1, 2 and 3 are respectively [0.5000, −0.7071]′, [0.5000,0.7071]′ and [0.7071, 0.0000]′. In addition, applying the foregoingexpression (4) to the feature vector of FIG. 4, the LSA feature vectorof the question from the user becomes [0.6542, 0]′.

[0021] Applying the foregoing expression (2) to the LSA feature vectorsobtained as set forth above, the similarity of the question of FIG. 3and each document of FIG. 1, become respectively, sim(document 1,question)=0.5774, sim(document 2, question)=0.5774, and sim(document 3,question)=1.0000. Thus, a result that the document 3 has the highestsimilarity to the question can be obtained. Considering a help systemapplication or the like utilizing computer networks, an answer statementof the document 3 registered in the document database will be returnedto the user who asked the question of FIG. 3.

[0022] For singular value decomposition, an algorithm proposed in“Matrix Computations”, The Johns Hopkins University Press, 1996, pp. 455to 457, is frequently used. In the report of “Journal of the AmericanSociety for Information Science” set forth above, there is a statementthat the value of the number of rows (or columns) n of the square matrixS is preferably about 50 to 150. In addition, in the foregoing report of“Behavior Research Methods, Instruments, & Computers”, it has beenindicated that better efficiency can be attained by pre-processing usingthe term frequency or document frequency instead of simply setting eachelement of the feature vector to 0 or 1 before performing LSA.

[0023] However, in the algorithm for singular value decompositionproposed in the foregoing “Matrix Computations”, memory space in theorder of the square of the number of index terms t (t²) is required atthe minimum. This is because a matrix of t×t is utilized forbidiagonalization of a matrix in the process of calculation of basisvectors spanning a feature space from a given term-document matrix. Theprior art is therefore not applicable to document database holding avery large number of terms and data. Furthermore, the prior art requirescomplicated operations of matrices irrespective of the number of data.

SUMMARY OF THE INVENTION

[0024] The present invention has been worked out in view of the problemsset forth above. In a first aspect of the present invention, a textmining method is provided for extracting features of documents using aterm-document matrix consisting of vectors corresponding to index termsrepresenting the contents of the documents. In the term-document matrix,contributions of the index terms to each document act on respectiveelements of the term-document matrix. The method comprises:

[0025] a basis vector calculating step of calculating a basis vectorspanning a feature space, in which mutually associated documents andterms are located in proximity with each other, based on a steepestdescent method minimizing a cost;

[0026] a feature extracting step of calculating a parameter fornormalizing features using the term-document matrix and the basisvector, and extracting the features on the basis of the parameter; and

[0027] a term-document matrix updating step of updating theterm-document matrix to a difference between the term-document matrix,to which the basis vector is not applied, and the term-document matrix,to which the basis vector is applied.

[0028] In a second aspect of the present invention, a text mining methodis provided for extracting features of documents wherein the cost isdefined as a second-order cost of the difference between theterm-document matrix, to which the basis vector is not applied, and theterm-document matrix, to which the basis vector is applied.

[0029] In a third aspect of the present invention, a text mining methodis provided for extracting features of documents wherein the basisvector calculating step comprises:

[0030] an initializing step of initializing a value of the basis vector;

[0031] a basis vector updating step of updating the value of the basisvector;

[0032] a variation degree calculating step of calculating a variationdegree of the value of the basis vector;

[0033] a judging step of making a judgment whether a repetition processis to be terminated or not using the variation of the basis vector; and

[0034] a counting step of counting the number of times of the repetitionprocess.

[0035] In a fourth aspect of the present invention, a text mining methodis provided for extracting features of documents wherein the basisvector updating step updates the basis vector using a current value ofthe basis vector, the term-document matrix and an updating ratiocontrolling the updating degree of the basis vector.

[0036] In a fifth aspect of the present invention, a text mining methodis provided for extracting features of documents wherein when all basisvectors and normalizing parameters required in extracting the featureshave been already obtained, the calculation of the normalizingparameters in the basis vector calculating step and the execution of thefeature extracting step are omitted. In addition, the feature extractingstep extracts the features using the basis vectors and the normalizingparameters that have been already obtained.

[0037] In a sixth aspect of the present invention, a text miningapparatus is provided for extracting features of documents using aterm-document matrix consisting of vectors corresponding to index termsrepresenting the contents of the documents. In the term-document matrix,contributions of the index terms to each document act on respectiveelements of the term-document matrix. The apparatus comprises:

[0038] basis vector calculating means for calculating a basis vectorspanning a feature space, in which mutually associated documents andterms are located in proximity with each other, based on a steepestdescent method minimizing a cost;

[0039] feature extracting means for calculating a parameter fornormalizing features using the term-document matrix and the basisvector, and extracting the features on the basis of the parameter; and

[0040] term-document matrix updating means for updating theterm-document matrix to a difference between the term-document matrix,to which the basis vector is not applied, and the term-document matrix,to which the basis vector is applied.

[0041] In a seventh aspect of the present invention, a text miningapparatus is provided for extracting features wherein the cost isdefined as a second-order cost of the difference between theterm-document matrix, to which the basis vector is not applied, and theterm-document matrix, to which the basis vector is applied.

[0042] In an eighth aspect of the present invention, a text miningapparatus is provided for extracting features of documents wherein thebasis vector calculating means comprises:

[0043] initializing means for initializing a value of the basis vector;

[0044] basis vector updating means for updating the value of the basisvector;

[0045] variation degree calculating means for calculating a variationdegree of the value of the basis vector;

[0046] judging means for making a judgment whether a repetition processis to be terminated or not using the variation of the basis vector; and

[0047] counting means for counting the number of times of the repetitionprocess.

[0048] In a ninth aspect of the present invention, a text miningapparatus is provided for extracting features of documents wherein thebasis vector updating means updates the basis vector using a currentvalue of the basis vector, the term-document matrix and an updatingratio controlling the updating degree of the basis vector.

[0049] In a tenth aspect of the present invention, a text miningapparatus is provided for extracting features of documents wherein whenall basis vectors and normalizing parameters required in extracting thefeatures have been already obtained, the calculation of the normalizingparameters in the basis vector calculating means and the execution ofthe feature extracting means are omitted. In addition, the featureextracting means extracts the features using the basis vectors and thenormalizing parameters that have been already obtained.

[0050] In an eleventh aspect of the present invention, a computerprogram product is provided for being executed in a text miningapparatus for extracting features of documents using a term-documentmatrix consisting of vectors corresponding to index terms representingthe contents of the documents. In the term-document matrix,contributions of the index terms act on respective elements of theterm-document matrix. The computer program product comprises:

[0051] a basis vector calculating step of calculating a basis vectorspanning a feature space, in which mutually associated documents andterms are located in proximity with each other, based on a steepestdescent method minimizing a cost;

[0052] a feature extracting step of calculating a parameter fornormalizing features using the term-document matrix and the basisvector, and extracting the features on the basis of the parameter; and

[0053] a term-document matrix updating step of updating theterm-document matrix to a difference between the term-document matrix,to which the basis vector is not applied, and the term-document matrix,to which the basis vector is applied.

[0054] The feature extracting apparatus disclosed in this specificationis constructed by defining a cost as a second-order function of adifference between the term-document matrix, to which the basis vectoris not applied, and the term-document matrix, to which the basis vectoris applied. The apparatus merely requires the following means:

[0055] a. basis vector calculating means for calculating a basis vectorapplying a steepest descent method to the cost;

[0056] b. feature extracting means for calculating a parameter fornormalizing features using the term-document matrix and the basisvector, and extracting the features on the basis of the parameter;

[0057] c. term-document matrix updating means for updating theterm-document matrix to the difference between the term-document matrix,to which the basis vector is not applied, and the term-document matrix,to which the basis vector is applied, in order to prevent redundantextraction of features; and

[0058] d. feature extraction control means for controlling execution ofrespective means.

[0059] The basis vector calculating means repeats calculation on thebasis of the input term-document matrix to finally derive one basisvector. The repetition process is terminated when the variation degreeof the basis vector becomes less than or equal to a predeterminedreference value.

[0060] The feature extracting means calculates a parameter fornormalizing the features on the basis of the input basis vector and theterm-document matrix, and extracts a feature for each document.

[0061] The term-document matrix updating means updates the term-documentmatrix on the basis of the input basis vector.

[0062] The feature extraction control means repeats execution of eachmeans until the number of the features defined by the user is satisfied.When the basis vectors and the normalizing parameters have been alreadycalculated, execution of the basis vector calculating means andcalculation of the normalizing parameters in the feature extractingmeans are omitted. Then, the feature extraction can be performed withthe construction incorporating the already obtained basis vectors andthe normalizing parameters.

[0063] According to the present invention, a text mining method forextracting features of documents using a term-document matrix consistingof vectors corresponding to index terms representing the contents of thedocuments, wherein contributions of the index terms act on respectiveelements of the term-document matrix, comprises the following steps:

[0064] i. a basis vector calculating step of calculating a basis vectorspanning a feature space, wherein mutually associated documents andterms are located in proximity with each other based on a steepestdescent method minimizing a cost;

[0065] ii. a feature extracting step of calculating a parameter fornormalizing features using the term-document matrix and the basisvector, and extracting the features on the basis of the parameter;

[0066] iii. a term-document matrix updating step of updating theterm-document matrix to a difference between the term-document matrix,to which the basis vector is not applied, and the term-document matrix,to which the basis vector is applied; and

[0067] iv. a feature extraction control step of controlling execution ofrespective steps.

[0068] Therefore, concerning feature extraction of documents in textmining, the features having the same nature as those obtained by LSA canbe extracted with smaller memory space than the apparatus or methodexecuting LSA. In addition, specific software or hardware for extractingthe features can be easily implemented.

[0069] The above and other objects, characteristics and advantages ofdifferent embodiments of the present invention will become more apparentfrom the following descriptions of embodiments thereof taken inconjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

[0070]FIG. 1 is an illustration showing one example of documentsregistered in a document database;

[0071]FIG. 2 is an illustration showing one example of a term-documentmatrix with taking Kanji terms appearing in the documents shown in FIG.1 as index terms;

[0072]FIG. 3 is an illustration showing one example of a questionactually input by a user;

[0073]FIG. 4 is an illustration showing a term-document matrix obtainedfrom the question in FIG. 3;

[0074]FIG. 5 is an illustration showing one embodiment of a featureextracting apparatus according to the present invention;

[0075]FIG. 6 is an illustration showing one example of a hardwareconstruction for implementing the present invention;

[0076]FIG. 7 is an illustration showing a structure of a term-documentmatrix data file;

[0077]FIG. 8 is an illustration showing a structure of a basis vectordata file, wherein the calculated basis vectors are stored;

[0078]FIG. 9 is an illustration showing a structure of a feature datafile;

[0079]FIG. 10 is an illustration showing a structure of a normalizingparameter data file;

[0080]FIG. 11 is a flowchart showing calculation of a basis vector inbasis vector calculating means; and

[0081]FIG. 12 is an illustration showing one example of an automaticdocument classifying system employing one embodiment of the featureextracting apparatus according to the present invention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0082]FIG. 5 is an illustration showing one embodiment of a featureextracting apparatus according to the present invention. As shown inFIG. 5, feature extraction control means 200 has term-document matrixupdating means 210, basis vector calculating means 220, featureextracting means 230. The reference numeral 100 denotes a term-documentmatrix data file, 300 denotes a basis vector data file, 400 denotes afeature data file, 450 denotes a normalizing parameter data file. In theterm-document matrix data file 100, a term-document matrix of collecteddocument data is stored. The term-document matrix updating means 210reads the term-document matrix from the term-document matrix data file100, and transfers the read term-document matrix to the basis vectorcalculating means 220 and the feature extracting means 230 withoutupdating the term-document matrix, in a first iteration process.

[0083] In a second and subsequent iteration processes, theterms-document matrix is updated on the basis of the basis vectortransferred from the basis vector calculating means 220. The result ofupdating is transferred to the basis vector calculating means 220 andthe feature extracting means 230. The basis vector calculating means 220calculates one basis vector through a repetition process based on theterm-document matrix transferred from the term-document matrix updatingmeans 210. Then, the degree of variation of the basis vector inrespective repetition is monitored for terminating the repetitionprocess when the degree of variation becomes less than or equal to apredetermined reference value. The basis vector calculating means 220stores the calculated basis vector in the basis vector data file 300 andin conjunction therewith, transfers the calculated basis vector to theterm-document matrix updating means 210 and the feature extraction means230. The feature extracting means 230 extracts one feature of eachdocument on the basis of the term-document matrix transferred from theterm-document matrix updating means 210 and the basis vector transferredfrom the basis vector calculating means 220. The result is stored in thefeature data file 400, and also the parameter for normalizing thefeatures is stored in the normalizing parameter data file 450.

[0084] Execution of the term-document matrix updating means 210, thebasis vector calculating means 220 and the feature extracting means 230is taken as one iteration process. Number of times of iterationprocesses will be indicated by suffix i, and number of featuresdesignated by the user is indicated by suffix n. The feature extractioncontrol means 200 repeats the process until a condition, i=n issatisfied. On the other hand, in a case where all of the required basisvectors and the required normalizing parameters have already beenobtained, execution of the basis vector calculation means 220 andcalculation of the normalizing parameters in the feature extractingmeans 230 may be omitted. Therefore, in such a case, the featureextraction control means 200 may be constructed with the term-documentmatrix updating means 210 incorporating the known basis vectors andnormalizing parameters, and with the feature extracting means 230.

[0085]FIG. 6 is an illustration showing one example of a hardwareconstruction for implementing the present invention. As shown in FIG. 6,the feature extracting apparatus includes the following components:

[0086] a central processing unit (CPU) 10 performing control for theoverall apparatus,

[0087] a memory 20 for storing the program and providing a temporarydata storage region required for executing the program,

[0088] a keyboard 30 for inputting data, and

[0089] a display 40 for generating a display screen.

[0090] The programs to be executed by the feature extraction controlmeans 200, the term-document matrix data file 100, the basis vector datafile 300, the feature data file 400, and the normalizing parameter datafile 450 are stored in the memory 20.

[0091] By taking this construction, the feature extraction is performedby CPU 10 receiving the command from the user through the keyboard 30, amouse pointing a desired position on the display 40, or the like. Itshould be noted that, in the example shown in FIG. 5, the featureextraction control means 200 has a stand-alone construction. However,the feature extraction control means 200 may be built-in other systems.

[0092]FIG. 7 is an illustration showing a structure of the term-documentmatrix data file. In FIG. 7, the reference numerals 101-1, 101-2, . . ., 101-d correspond to t-dimensional term-document data A consisting of din number of data. Here, X=[x₁, x₂, . . . x_(d)], x_(j)=[x_(j1), x_(j2),. . . x_(jt)]′ are defined to express the term-document data A with at×d matrix X.

[0093]FIG. 8 is an illustration showing a structure of the basis vectordata file storing the calculated basis vectors. In FIG. 8, the referencenumerals 301-1, 301-2, . . . 301-n correspond to t-dimensional basisvector data B consisting of n in number of data. The (i)th element 301-icorresponds to an output value of the basis vector calculating means 220in the (i)th iteration process in FIG. 5. In the following disclosure,this element is expressed by a t×1 column vector w_(i)=[w_(i1), w_(i2),. . . , w_(it)]′.

[0094]FIG. 9 is an illustration showing a structure of the feature datafile. In FIG. 9, the reference numerals 401-1, 401-2, . . . , 401-ncorrespond to d-dimensional feature data C consisting of n in number ofdata. The (i)th element 401-i corresponds to an output value of thefeature by the feature extraction means 230 in the (i)th iterationprocess in FIG. 5. This element is expressed by an 1×d row vectory_(i)=[y_(il), y_(i2), . . . , y_(id)].

[0095]FIG. 10 is an illustration showing a structure of the normalizingparameter data file. In FIG. 10, the reference numerals 451-1, 451-2, .. . , 451-n correspond to normalizing parameter data D consisting of nin number of data. The (i)th element 451-i corresponds to an outputvalue of the normalizing parameter by the feature extracting means 230in the (i)th iteration process in FIG. 5.

[0096] Using the foregoing definitions, an implementation of featureextraction in the shown embodiment will be explained. The term-documentmatrix updating means 210 reads out X from the term-document matrix datafile 100 only when i=1, namely in the first iteration process, to storein a t×d matrix E without performing any arithmetic operation.Accordingly, E=[e₁, e₂, . . . , e_(d)], e_(j)=[e_(j1), e_(j2), . . . ,e_(jt)]=[x_(j1), x_(j2), . . . , x_(jt)]′. In order to prevent redundantextraction of the features extracted in the preceding iterationprocesses, E is updated in the (i)th iteration using the current valueand the basis vector calculated in the immediately preceding iterationprocess. The result of updating is transferred to the basis vectorcalculating means 220. A value of E in the (i)th iteration, E(i), willbe expressed by the following expression (5): $\begin{matrix}{{E(i)} = \left\{ \begin{matrix}{X,} & {{{for}\quad i} = 1} \\{{{E\left( {i - 1} \right)} - {w_{i - 1}\left( {w_{i - 1}^{\prime}{E\left( {i - 1} \right)}} \right)}},} & {otherwise}\end{matrix} \right.} & (5)\end{matrix}$

[0097] Here, E(i)=[e_(i)(i), e₂(i), . . . , e_(d)(i)], each elemente_(j)(i) of E(i) is defined by e_(j)(i)=[e_(j1)(i), e_(j2)(i), . . . ,e_(jt)(i)]′. Namely, when i>2, the term-document matrix is updated to adifference derived by subtracting the term-document matrix, to which thebasis vector is applied, from the term-document matrix, to which thebasis vector is not applied.

[0098]FIG. 11 is a flowchart showing calculation of the basis vector inthe basis vector calculating means. In FIG. 11, a value of w_(i) in the(k)th repetition is expressed by w_(i)(k)=[w_(i1)(k), w_(i2)(k), . . . ,w_(it)(k)]′. At first, at step S500, the suffix k is initialized to 1.Subsequently, the process is advanced to step S510 to initializerespective element of w_(i)(1) with an arbitrary value between −C to C.Here, the value of C may be a positive small value, such as C=0.01. Atstep S520, in order to calculate the basis vector spanning a featurespace where mutually associated documents and terms are located inproximity with each other, a second-order cost expressed by thefollowing expression (6) is provided. $\begin{matrix}{\frac{1}{2d}{\sum\limits_{m = 1}^{d}{\sum\limits_{l = 1}^{t}\left( {{e_{l\quad m}(i)} - {w_{li}{\overset{\sim}{y}}_{im}}} \right)^{2}}}} & (6)\end{matrix}$

[0099] Here, “terms are placed in proximity” means that the positions ofthe terms are close with each other within a feature space, and“documents are placed in proximity” means that the positions of termsincluded in respective documents are close in the feature space. On theother hand, a cost means an object to be minimized. In the shownembodiment, the cost is defined as a second-order function of thedifference between the term-document matrix, to which the basis vectoris not applied, and the term-document matrix, to which the basis vectoris applied, as expressed by the expression (6). Here,

{overscore (y)}_(1m)

[0100] is the (m)th element of a 1×d vector {overscore (y)}₁ which isdefined as follows:

{overscore (y)}₁=[{overscore (y)}_(i1), {overscore (y)}_(i2), . . . ,{overscore (y)}_(id)]=w_(i)′E(i)  (7)

[0101] For the cost, the steepest descent method is applied to updatethe value of w_(i) as expressed by the following expression (8).$\begin{matrix}{{w_{i}\left( {k + 1} \right)} = {{w_{i}(k)} + {\frac{\mu_{i}(k)}{d}\left( {{E(i)} - {{w_{i}(k)}{z_{i}(k)}}} \right){z_{i}(k)}^{\prime}}}} & (8)\end{matrix}$

[0102] Here, μ_(i)(k) is an update ratio controlling the degree ofupdating in the (k)th repetition, which is initialized by a positivesmall value when k is 1, such as μ_(i)(1)=0.1. Every time of incrementof k, the value is decreased gradually. In the alternative, it is alsopossible to set the value at a constant value irrespective of k. On theother hand, z_(i)(k) is defined as follows:

z_(i)(k)=w_(i)(k)′E(i)  (9)

[0103] At step S530, δ_(i)(k) indicating the degree of variation ofw_(i) is derived as follows: $\begin{matrix}{{\delta_{i}(k)} = \sqrt{\sum\limits_{j = 1}^{i}\left( {{w_{ji}\left( {k + 1} \right)} - {w_{ji}(k)}} \right)^{2}}} & (10)\end{matrix}$

[0104] At step S540, a judgment is made whether the process is to beterminated or not on the basis of the value of δ_(i)(k). As a result ofthe judgment, if termination is determined, the process is advanced tostep S560, and otherwise, the process is advanced to step S550. Here, inFIG. 11, β_(i) is a positive small value, such as β_(i)=1×10⁻⁶.

[0105] At step S550, the value of the counter k is incremented by 1.Then, the process is returned to step S520. At step S560, w_(i) isstored as the (i)th data of the basis vector data file 300. At the sametime, w_(i) is transferred to the term-document matrix updating means210 and the feature extracting means 230. In the feature extractingmeans 230, the feature y_(i) and the normalizing parameter p_(i) arecalculated in the following manner.

y_(i)={overscore (y)}_(i)/p₁  (11)

[0106] Here, p_(i) is defined as follows: $\begin{matrix}{p_{i} = \sqrt{\sum\limits_{j = 1}^{d}{\overset{\sim}{y}}_{ij}^{2}}} & (12)\end{matrix}$

[0107] The feature y_(i) and the normalizing parameter p_(i) are storedrespectively in the feature data file 400 and the normalizing parameterdata file 450 as the (i)th data.

[0108]FIG. 12 is an illustration showing one example of an automaticdocument classifying system employing the shown embodiment of thefeature extracting apparatus. In FIG. 12, the reference numeral 601denotes term-document matrix calculating means, 602 denotes classifyingmeans. The classifying means 602 may be implemented by a methoddisclosed in “Journal of Intelligent and Fuzzy Systems”, published on1993, Vol. 1, No. 1, Pages 1 to 25.

[0109] The document data stored in the document database E is taken inthe automatic document classifying system 600. In the automatic documentclassifying system 600, a term-document matrix is derived in theterm-document matrix calculating means 601. The result of calculation ofthe term-document matrix is transferred to the feature extractioncontrol means 200. The feature extraction control means 200 extracts thefeatures from the received term-document matrix. The extracted result isoutput to the classifying means 602. In the classifying means 602, theresult of classification is output on the basis of the input features.

[0110] To evaluate the present invention, feature extraction of actualdocument data related to an entrance examination system was performed.It has been confirmed that the present invention could extract thefeatures of the same nature as those extracted using the conventionalLSA.

[0111] Next, concerning the size of the memory space, in a typicallypractical case where the number of terms t is significantly greater thannumber of documents d (t>>d), the conventional LSA requires in the orderof t² of the memory size, the present invention merely requires thememory size in the order of t·d for calculating respective basisvectors. Furthermore, in order to realize the prior art, a complicatedmatrix operation apparatus is required. The system according to theinvention, however, can be easily realized with an apparatus thatperforms simple arithmetic operations. Namely, according to the presentinvention, the LSA feature extraction can be performed using a smallermemory space and a simpler program. In addition, this simple program maybe loaded in a digital signal processor (DSP). Therefore, a specificchip for feature extraction can be produced easily.

[0112] Hereinafter, the results of respective means executing the shownembodiment of the feature extracting apparatus for the documents of FIG.1 and the question of FIG. 3 will be shown.

[0113] A. Documents of FIG. 1

[0114] First, let X denote the term-document matrix of FIG. 2.

[0115] I. First Iteration in Feature Extraction Control Means 200 (i=1)

[0116] According to the foregoing expression (5), the term-documentmatrix updating means 210 outputs E(1) expressed by the followingexpression to the basis vector calculating means 220 and the featureextracting means 230. ${E(1)} = \begin{bmatrix}1 & 0 & 0 \\1 & 0 & 0 \\1 & 0 & 0 \\1 & 0 & 1 \\1 & 0 & 1 \\0 & 1 & 0 \\0 & 1 & 0 \\0 & 1 & 1 \\0 & 1 & 1 \\0 & 1 & 0 \\0 & 0 & 1\end{bmatrix}$

[0117] In the basis vector calculating means 220, initialization isperformed with setting the basis vector w_(i)(1) at [0.0100, −0.0100,0.0100, −0.0100, 0.0100, −0.0100, 0.0100, −0.0100, 0.0 100, −0.0100,0.0100]′, μ₁ at a fixed value 0.1, β₁ at 1×10⁻⁶. The calculation shownin FIG. 11 is performed for a hundred thirty-two times. Then, the basisvector w₁=[0.1787, 0.1787, 0.1787, 0.4314, 0.4314, 0.1787, 0.1787,0.4314, 0.4314, 0.1787, 0.2527]′is stored in the basis vector data file300, and output to the feature extracting means 230 and theterm-document matrix updating means 210.

[0118] First Repetition in Basis Vector Calculating Means 220 (k=1)

[0119] From the foregoing expression (8),

w₁(2)=[0.0103 −0.0097, 0.0103, 0.0093, 0.0107, −0.0103, 0.0097, −0.0100,0.0100, −0.0103, 0.0103]′

w ₁(2)−w ₁(1)=10⁻³×[0.3332, 0.3334, 0.3332, 0.6668, 0.6666, −0.3332,−0.3334, 0.0001, −0.0001, −0.3332, 0.3332]′

δ₁(1)=0.0103

[0120] Second Repetition in Basis Vector Calculating Means 220 (k=2)

[0121] From the foregoing expression (8),

w₁(3)=[0.0107, −0.0093, 0.0107, −0.0085, 0.0115, −0.0107, 0.0093,−0.0100, 0.0100, −0.0107, 0.0107]′

w ₁(3)−w ₁(2)=10⁻³×[0.4110, 0.4112, 0.4110, 0.8001, 0.7998, −0.3665,−0.3668, 0.0224, 0.0221, −0.3665, 0.3887]′

δ₁(2)=0.0015

[0122] —syncopated—

[0123] A Hundred Thirty-Second Repetition in Basis Vector CalculatingMeans 220 (k=132)

[0124] From the foregoing expression (8),

w₁(133)=[0.1787, 0.1787, 0.1787, 0.4314, 0.4314, 0.1787, 0.1787, 0.4314,0.4314, 0.1787, 0.2527]′

w ₁(133)−w ₁(132)=10⁻⁶×[−0.3020, −0.3020, −0.3020, −0.3020, −0.3020,0.3020, 0.3020, 0.3020, 0.3020, 0.3020, 0.0000]′

δ₁(132)=9.5500×10⁻⁷

[0125] In the feature extracting means 230, the operations shown in theexpressions (11) and (12) are performed for outputting:

y₁=[0.5000, 0.5000, 0.7071]

[0126] and

p₁=2.7979

[0127] to the feature data file 400 and the normalizing parameter datafile 450.

[0128] II. Second Iteration in Feature Extraction Control Means 200(i=2)

[0129] In the term-document matrix updating means 210, from theforegoing expression (5), E(2) expressed as follows is output to thebasis vector calculating means 220 and the feature extracting means 230:${E(2)} = \begin{bmatrix}0.7500 & {- 0.2500} & {- 0.3536} \\0.7500 & {- 0.2500} & {- 0.3536} \\0.7500 & {- 0.2500} & {- 0.3536} \\03964 & {- 0.6036} & 0.1464 \\0.3964 & {- 0.6036} & 0.1464 \\{- 0.2500} & 0.7500 & {- 0.3535} \\{- 0.2500} & 0.7500 & {- 0.3535} \\{- 0.6036} & 0.3965 & 0.1465 \\{- 0.6036} & 0.3965 & 0.1465 \\{- 0.2500} & 0.7500 & {- 0.3535} \\{- 0.3536} & {- 0.3535} & 0.5000\end{bmatrix}$

[0130] In the basis vector calculating means 220, initialization isperformed with setting the basis vector w₂(1) at [0.0100, −0.0100,0.0100, −0.0100, 0.0100, −0.0100, 0.0100, −0.0100, 0.0100, −0.0100,0.0100]′, μ_(2 at a fixed value) 0.1, β_(2 at) 1×10⁻⁶. The calculationshown in FIG. 11 is performed for a hundred nineteen times. Then, thebasis vector w₂ =[0.3162, 0.3162, 0.3162, 0.3162, 0.3162, −0.3162,−0.3162, −0.3162, - 0.3162, −0.3162, 0.0000]′ is stored in the basisvector data file 300, and output to the feature extracting means 230 andthe term-document matrix updating means 210.

[0131] First Repetition in Basis Vector Calculating Means 220 (k=1)

[0132] From the foregoing expression (8),

w₂(2)=[0.0102, −0.0098, 0.0102, −0.0096, 0.0104, −0.0105, 0.0095,−0.0103, 0.0097, −0.0105, 0.0102]′

w ₂(2)−w ₂(1)=10⁻³×[0.2154, 0.2156, 0.2154, 0.3822, 0.3821, −0.4511,−0.4513, −0.2844, −0.2846, −0.4511, 0.1666]′

δ₂(1)=0.0011

[0133] Second Repetition in Basis Vector Calculating Means 220 (k=2)

[0134] From the foregoing expression (8),

w₂(3)=[0.0105, −0.0095, 0.0105, −0.0092, 0.0108, −0.0110, 0.0090,−0.0106, 0.0094, −0.0110, 0.0103]′

w ₂(3)−w ₂(2)=10⁻³×[0.2624, 0.2626, 0.2624, 0.4413, 0.4411, −0.5152,−0.5154, −0.3364, −0.3366, −0.5152, 0.1786]′

δ₂₍2)=0.0013

[0135] —syncopated—

[0136] A Hundred Nineteenth Repetition in Basis Vector Calculating Means220 (k=119)

[0137] From the foregoing expression (8),

w₂(120)=[0.3162, 0.3162, 0.3162, 0.3162, 0.3162, −0.3162, −0.3162,−0.3162, −0.3162, 0.0000]′

w ₂(120)−w ₂(119)=10⁻⁶×[0.3327, 0.3333, 0.3327, −0.1375, −0.1381,0.3332, 0.3326, −0.1377, −0.1383, 0.3332, −0.4712]′

δ₂₍119)=9.8141×10⁻⁷

[0138] In the _(feature) extracting means 230, the operations shown inthe expressions (11) and (12) are performed for outputting:

y₂=[0.7071, −0.7071, −0.0000]

[0139] and

p₂=2.2361

[0140] to the feature data file 400 and the normalizing parameter datafile 450.

[0141] From the results set forth above, the feature vectors of thedocuments 1, 2 and 3 in FIG. 1 are respectively [0.5000, 0.7071]′,[0.5000, −0.7071], [0.7071, −0.0000]. Comparing these with the featuresof the LSA of respective documents shown in the explanation of the priorart, the second element of each vector is of opposite sign but has thesame absolute value. Accordingly, concerning calculation of similarityin the expression (2), they have the same nature as the features of LSA.

[0142] B. Question of FIG. 3

[0143] Here, let us use the basis vectors stored in the basis vectordata file 300 and the normalizing parameters stored in the normalizingparameter data file 450 during extraction of the features of thedocuments of FIG. 1. Thereby, execution of the basis vector calculatingmeans 220 and calculation of the normalizing parameter in the featureextracting means 230 are omitted. Let X denote the term-document matrixof FIG. 4.

[0144] I. First Iteration in Feature Extracting Means 200 (i=1)

[0145] In the term-document matrix updating means 210, E(1) expressed asfollows from the foregoing expression (5) is output to the featureextracting means 230. ${E(1)} = \begin{bmatrix}0 \\0 \\1 \\1 \\1 \\1 \\1 \\0 \\1 \\0 \\0\end{bmatrix}$

[0146] In the feature extracting means 230, arithmetic operationaccording to the foregoing expressions (11) and (12) is performed usingthe feature vector w₁ and the normalizing parameter p₁ obtained uponextraction of the features of the documents of FIG. 1 to output

y₁=[0.6542]

[0147] to the feature data file 400.

[0148] II. Second Iteration in Feature Extraction Control Means 200(i=2)

[0149] In the term-document matrix updating means 210, using the featurevector w₁ obtained upon performing feature extraction of the documentsshown in FIG. 1, from the foregoing equation (5), E(2) expressed asfollows is output to the feature extracting means 230.${E(2)} = \begin{bmatrix}{- 0.3271} \\{- 0.3271} \\0.6729 \\0.2103 \\0.2103 \\0.6729 \\0.6729 \\{- 0.7897} \\0.2103 \\{- 0.3271} \\{- 0.4626}\end{bmatrix}$

[0150] In the feature extracting means 230, arithmetic operationaccording to the foregoing expressions (11) and (12) is performed usingthe feature vector w₂ and the normalizing parameter p₂ obtained uponextraction of the features of the documents of FIG. 1 to output

y₂=[−0.0000]

[0151] to the feature data file 400.

[0152] From the result set forth above, the feature vector of thequestion of FIG. 3 becomes [0.6542, −0.0000]′, comparing the valueexplained in the prior art, the second element has the same absolutevalue.

[0153] The present invention has been described in detail with respectto preferred embodiments. It will now be apparent from the foregoing tothose skilled in the art that changes and modifications may be madewithout departing from the invention in its broader aspect. It is theintention, therefore, in the apparent claims to cover all such changesand modifications as fall within the true spirit of the invention.

What is claimed is:
 1. A text mining method for extracting features ofdocuments using a term-document matrix consisting of vectorscorresponding to index terms representing the contents of the documents,wherein contributions of the index terms act on respective elements ofthe term-document matrix, said method comprising: a basis vectorcalculating step of calculating a basis vector spanning a feature space,in which mutually associated documents and terms are located inproximity with each other, based on a steepest descent method minimizinga cost; a feature extracting step of calculating a parameter fornormalizing the features using the term-document matrix and the basisvector and extracting the features on the basis of the parameter; and aterm-document matrix updating step of updating the term-document matrixto a difference between the term-document matrix, to which the basisvector is not applied, and the term-document matrix, to which the basisvector is applied.
 2. A text mining method for extracting features ofdocuments as claimed in claim 1, wherein the cost is defined as asecond-order cost of the difference between the term-document matrix, towhich the basis vector is not applied, and the term-document matrix, towhich the basis vector is applied.
 3. A text mining method forextracting features of documents as claimed in claim 2, wherein saidbasis vector calculating step comprises: an initializing step ofinitializing a value of the basis vector; a basis vector updating stepof updating the value of the basis vector; a variation degreecalculating step of calculating a variation degree of the value of thebasis vector; a judging step of making a judgment whether a repetitionprocess is to be terminated or not using the variation degree of thebasis vector; and a counting step of counting the number of times ofsaid repetition process.
 4. A text mining method for extracting featuresof documents as claimed in claim 3, wherein said basis vector updatingstep updates the basis vector using a current value of the basis vector,the term-document matrix and an updating ratio controlling the updatingdegree of the basis vector.
 5. A text mining method for extractingfeatures of documents as claimed in claim 4, wherein, when all basisvectors and normalizing parameters required in extracting the featureshave been already obtained, the calculation of normalizing parameters insaid basis vector calculating step and the execution of said featureextracting step are omitted, and said feature extracting step extractsthe features using the basis vectors and the normalizing parameters thathave been already obtained.
 6. A text mining method for extractingfeatures of documents as claimed in claim 3, wherein, when all basisvectors and normalizing parameters required in extracting the featureshave been already obtained, the calculation of normalizing parameters insaid basis vector calculating step and the execution of said featureextracting step are omitted, and said feature extracting step extractsthe features using the basis vectors and the normalizing parameters thathave been already obtained.
 7. A text mining method for extractingfeatures of documents as claimed in claim 2, wherein, when all basisvectors and normalizing parameters required in extracting the featureshave been already obtained, the calculation of normalizing parameters insaid basis vector calculating step and the execution of said featureextracting step are omitted, and said feature extracting step extractsthe features using the basis vectors and the normalizing parameters thathave been already obtained.
 8. A text mining method for extractingfeatures of documents as claimed in claim 1, wherein said basis vectorcalculating step comprises: an initializing step of initializing a valueof the basis vector; a basis vector updating step of updating the valueof the basis vector; a variation degree calculating step of calculatinga variation degree of the value of the basis vector; a judging step ofmaking a judgment whether a repetition process is to be terminated ornot using the variation degree of the basis vector; and a counting stepof counting the number of times of said repetition process.
 9. A textmining method for extracting features of documents as claimed in claim8, wherein said basis vector updating step updates the basis vectorusing a current value of the basis vector, the term-document matrix andan updating ratio controlling the updating degree of the basis vector.10. A text mining method for extracting features of documents as claimedin claim 9, wherein, when all basis vectors and normalizing parametersrequired in extracting the features have been already obtained, thecalculation of normalizing parameters in said basis vector calculatingstep and the execution of said feature extracting step are omitted, andsaid feature extracting step extracts the features using the basisvectors and the normalizing parameters that have been already obtained.11. A text mining method for extracting features of documents as claimedin claim 8, wherein, when all basis vectors and normalizing parametersrequired in extracting the features have been already obtained, thecalculation of normalizing parameters in said basis vector calculatingstep and the execution of said feature extracting step are omitted, andsaid feature extracting step extracts the features using the basisvectors and the normalizing parameters that have been already obtained.12. A text mining method for extracting features of documents as claimedin claim 1, wherein, when all basis vectors and normalizing parametersrequired in extracting the features have been already obtained, thecalculation of normalizing parameters in said basis vector calculatingstep and the execution of said feature extracting step are omitted, andsaid feature extracting step extracts the features using the basisvectors and the normalizing parameters that have been already obtained.13. A text mining apparatus for extracting features of documents using aterm-document matrix consisting of vectors corresponding to index termsrepresenting the contents of the document, wherein contributions of theindex terms act on respective elements of the term-document matrix, saidapparatus comprising: basis vector calculating means for calculating abasis vector spanning a feature space, in which mutually associateddocuments and terms are located in proximity with each other, based on asteepest descent method minimizing a cost; feature extracting means forcalculating a parameter for normalizing the features using theterm-document matrix and the basis vector and extracting the features onthe basis of the parameter; and term-document matrix updating means forupdating the term-document matrix to a difference between theterm-document matrix, to which the basis vector is not applied, and theterm-document matrix, to which the basis vector is applied.
 14. A textmining apparatus for extracting features of documents as claimed inclaim 13, wherein the cost is defined as a second-order cost of thedifference between the term-document matrix, to which the basis vectoris not applied, and the term-document matrix, to which the basis vectoris applied.
 15. A text mining apparatus for extracting features ofdocuments as claimed in claim 14, wherein said basis vector calculatingmeans comprises: initializing means for initializing a value of thebasis vector; basis vector updating means for updating the value of thebasis vector; variation degree calculating means for calculating avariation degree of the value of the basis vector; judging means formaking a judgment whether a repetition process is to be terminated ornot using the variation degree of the basis vector; and counting meansfor counting the number of times of said repetition process.
 16. A textmining apparatus for extracting features of documents as claimed inclaim 15, wherein said basis vector updating means updates the basisvector using a current value of the basis vector, the term-documentmatrix and an updating ratio controlling the updating degree of thebasis vector.
 17. A text mining apparatus for extracting features ofdocuments as claimed in claim 16, wherein, when all the basis vectorsand normalizing parameters required in extracting the feature have beenalready obtained, the calculation of normalizing parameters by saidbasis vector calculating means and the execution of said featureextracting means are omitted, and said feature extracting means extractsthe features using the basis vectors and the normalizing parameters thathave been already obtained.
 18. A text mining apparatus for extractingfeatures of documents as claimed in claim 15, wherein, when all basisvectors and normalizing parameters required in extracting the featureshave been already obtained, the calculation of normalizing parameters bysaid basis vector calculating means and the execution of said featureextracting means are omitted, and said feature extracting means extractsthe features using the basis vectors and the normalizing parameters thathave been already obtained.
 19. A text mining apparatus for extractingfeatures of documents as claimed in claim 14, wherein, when all basisvectors and normalizing parameters required in extracting the featureshave been already obtained, the calculation of normalizing parameters bysaid basis vector calculating means and the execution of said featureextracting means are omitted, and said feature extracting means extractsthe features using the basis vectors and the normalizing parameters thathave been already obtained.
 20. A text mining apparatus for extractingfeatures of documents as claimed in claim 13, wherein said basis vectorcalculating means comprises: initializing means for initializing a valueof the basis vector; basis vector updating means for updating the valueof the basis vector; variation degree calculating means for calculatinga variation degree of the value of the basis vector; judging means formaking a judgment whether a repetition process is to be terminated ornot using the variation degree of the basis vector; and counting meansfor counting the number of times of said repetition process.
 21. A textmining apparatus for extracting features of documents as claimed inclaim 20, wherein said basis vector updating means updates the basisvector using a current value of the basis vector, the term-documentmatrix and an updating ratio controlling the updating degree of thebasis vector.
 22. A text mining apparatus for extracting features ofdocuments in text mining as claimed in claim 21, wherein, when all basisvectors and normalizing parameters required in extracting the featurehave been already obtained, the calculation of normalizing parameters bysaid basis vector calculating means and the execution of said featureextracting means are omitted, and said feature extracting means extractsthe features using the basis vectors and the normalizing parameters thathave been already obtained.
 23. A text mining apparatus for extractingfeatures of documents as claimed in claim 20, wherein, when all basisvectors and normalizing parameters required in extracting the featureshave been already obtained, the calculation of normalizing parameters bysaid basis vector calculating means and the execution of said featureextracting means are omitted, and said feature extracting means extractsthe features using the basis vectors and the normalizing parameters thathave been already obtained.
 24. A text mining apparatus for extractingfeatures of documents as claimed in claim 13, wherein, when all basisvectors and normalizing parameters required in extracting the featureshave been already obtained, the calculation of normalizing parameters bysaid basis vector calculating means and the execution of said featureextracting means are omitted, and said feature extracting means extractsthe features using the basis vectors and the normalizing parameters thathave been already obtained.
 25. A computer program product for beingexecuted in a text mining apparatus for extracting features of documentsusing a term-document matrix consisting of vectors corresponding toindex terms representing the contents of the documents, whereincontributions of the index terms act on respective elements of theterm-document matrix, the computer program product comprising: basisvector calculating step of calculating a basis vector spanning a featurespace, in which mutually associated documents and terms are located inproximity with each other, based on a steepest descent method minimizinga cost; feature extracting step of calculating a parameter fornormalizing the features using the term-document matrix and the basisvector and extracting the features on the basis of the parameter; andterm-document matrix updating step of updating the term-document matrixto a difference between the term-document matrix, to which the basisvector is not applied, and the term-document matrix, to which the basisvector is applied.